The subject review visit

Who reviews?

Subject reviews are carried out by a team of subject specialist reviewers led by a review coordinator (who will not be a subject specialist).

According to the QAA it is the subject specialist reviewers’ main responsibility “to gather evidence and to make judgements on the quality of education provided”. Subject specialist reviewers are drawn primarily from within HE and are trained by the QAA before their first review visit. The number of subject reviewers in each team reflects the size, range and complexity of the education provided. As far as possible, the Agency matches the collective expertise of the team with the broad law specialisms of the department/school.

Using its register of reviewers and the criteria for composing teams outlined below, the Agency will propose a subject review team to an institution before the review starts. Account is taken of conflicts of interest declared by subject reviewers. If a review is combined with activity of a professional or statutory body, the requirements of that body will also be considered. Institutions are invited to comment on the composition of teams and to confirm their agreement in writing to the Agency within four weeks of notification. Any concerns about the suitability of reviewers should be discussed with Agency officers as soon as possible after notification and, if not resolved satisfactorily, put in writing to the Agency.

Experience to date has shown that it is exceptionally helpful to be able to call on the experience of a subject specialist reviewer when preparing for subject review. Participation as a reviewer on subject review visits to other institutions provides a range of useful insights into both the operation of the methodology and others’ ingredients for success.

The review coordinator

The Agency expects the review coordinator for each review to work closely with the subject review facilitator and/or the department/school review coordinator within an institution. The review coordinator will also make initial contact with the subject specialist reviewers, will liaise with both the Agency and the institution about the arrangements for the first visit of the review team to the institution, will prepare for the initial team meeting and allocate reviewer responsibilities, and will be responsible for channeling all requests for documentation from reviewers to the department/school. You should therefore look upon the review coordinator as the main point of contact for the review. The review coordinator is also responsible for keeping the QAA and the department/school informed about the progress of any particular review.

For its part, an institution will want to receive information about the likely pattern of the review and will want to arrange specific times when reviewers will visit. In order to facilitate this, the QAA is expecting review coordinators to agree a tentative schedule with subject specialist reviewers before or at the initial team meeting and to convey this to the department/school. The Agency will also ask teams to agree a tentative date for completion of each review and will expect this to be communicated to the institution as a matter of priority.

The subject review facilitator

Your institution will have several trained subject review facilitators, one of whom should be assigned to work closely with the department/school prior to and throughout the period of the review and will attend most of the meetings which take place during the review.

The purpose of the role is to provide effective liaison between the team of reviewers and department/school staff and to ensure that the team obtains accurate and comprehensive information about the educational provision and its institutional context. Facilitators are briefed for their role by the Agency. The formal responsibilities of the facilitator are set out in Annex F of the QAA handbook.

Team function for subject review

Subject specialist reviewers focus their attention on the subject and only address institutional matters when they have a direct bearing on the student learning process. It is, however, important that review co-ordinators ensure that matters related to institutional function which come to their team’s attention are reported, thereby making them available to the reviewers who carry out institutional review.

Subject specialist reviewers assume a collective responsibility for gathering and verifying evidence in relation to academic standards, but may concentrate individually on specific matters in relation to the quality of learning opportunities. All judgements are, however, made collectively.

General approach

Reviews are intended to be conducted in a spirit of dialogue and cooperation between the institutions, their subject staff, and the review teams. Reviewers must be able to gather sufficient evidence on the subject provision to allow them to test statements made in the self-evaluation, and to form robust judgements on the quality and standards of the provision.

At its first meeting, the review team will consider:

• self-evaluation and any other documentation supplied by the department/school prior to the review

• scope and nature of the provision
• main matters for review and judgement
• role of the facilitator in relation to the conduct of the review
• allocation of individual responsibilities amongst the members of the team
• programme activities, both on- and off-site, required for the review
• pattern and timing of visits to the department/school

The review team will then hold an initial meeting with the department/school. The department/school may wish to make a brief presentation to introduce the provision to be reviewed, and to describe any developments since the self-evaluation was prepared. The review coordinator will remind both the team and the institutional representatives of the method and protocols of review. eviewers will agree an outline programme for the review and will establish:

• range of student work which can be made available for scrutiny, and the extent to which this constitutes a representative sample of student achievement
• nature of relevant documentation held by the department/school and its availability for scrutiny by reviewers
• range and timing of internal quality assurance ‘events’, such as programme committees, faculty boards (or equivalent) or examination boards, which might provide documentary evidence and/or be attended (by agreement with the department/school) by reviewers

• timing of any related visits by the professional body
• probable agenda and timing of meetings with academic staff, students and former students
• other practical arrangements for the review

The review team will not normally ask for specially prepared documentation, other than the self-evaluation. It will endeavour to make use of existing documentation used for internal processes related to quality and standards.

The pattern of review activity over a period will enable material to be requested well in advance of any visit to the institution. In most cases, subject providers will be able to identify appropriate samples from work completed by students in the current academic year or from materials kept routinely for examination purposes.

Testing the self-evaluation and gathering evidence

The review method provides a structure for the self-evaluation, and for the visits, judgements and reports made by reviewers. It involves addressing:

• subject provision and aims
• learning outcomes
• curricula and assessment
• quality of learning opportunities
• student achievement
• maintenance and enhancement of quality and standards

As may be seen in the previous section the self-evaluation is written to this framework. The QAA’s aide-memoire (Annex E of the handbook) for subject review is structured similarly.

Documentary evidence

This includes internal reports from committees, boards and individual staff with relevant responsibilities, and external reports from examiners, employers, validating and accrediting bodies.

Emerging judgements are refined and tested against as wide a range of evidence as possible; for example, the views expressed in meetings by staff or by students are tested against the documentation provided.

See also the section of this guide on preparation for the visit

Whenever and however points of concern or requests for clarification are raised by the reviewers, it is vital that you seek to provide a response and, wherever possible, documentary evidence to ensure that:

• reviewers do not retain a misapprehension about the nature of the provision
• reviewers are able to consider additional evidence which will prevent them from reaching a negative conclusion in relation to one or more aspects of provision

Your subject review facilitator is expected to play an important role in pointing reviewers to documents they might have not taken fully into account but _the prime responsibility lies with the department/school to provide the evidence to confirm that aims are being met_.

Some further notes on documentation:

• where informal interactions among a small group of staff have been identified as important in your self-evaluation document, the reviewers will look for evidence of the effectiveness of such informal arrangements and will expect to see some references to such occasions in formal documentation
• communications between the department/school’s subject review co-ordinator and central services providers will need to be well developed prior to the review and maintained (at least via e-mail) over the period of the visit in order that additional documentation can easily be obtained from them if necessary
• every piece of assessed work which reviewers will see must
  either include clear feedback to the student or have attached a brief statement about how the feedback was provided to the student(s)
• reports of the annual monitoring or review of courses will have to be included in material provided for reviewers and it is vital that these documents confirm that the department/school has, at the very least, followed whatever the institutional requirements are
• reviewers will wish to see formal responses to external examiners’ reports; it is crucial that, in addition to responding to the issues raised in external examiners’ reports, departments/schools routinely respond directly to their externals (in writing and in good time) advising them of the actions which have been taken in response to the comments made in reports. This is now a general expectation articulated in the QAA code of practice.

Observation of teaching

Subject reviewers may not need to make direct observations of teaching where a department/school can provide evidence of good quality delivery. Such evidence is likely to come from internal peer review, from student questionnaires and other arrangements for gathering feedback, from the deployment of learning resources, and from student performance in assessments.

Direct observation of teaching will be required if:

• there are issues that reviewers feel would be best addressed by such observation
• observation might help confirm a judgement about exemplary provision
• there is insufficient other evidence that effective delivery is being achieved
• there are indications that the learning opportunities for students are less than satisfactory

A note on observation of teaching is at Appendix VI

Meetings with students, former students and employers

Meetings with students enable reviewers to establish student views on the issues being considered. These meetings provide an opportunity not only to hear the direct views of those present, but also to establish more generally whether there are effective arrangements for student feedback and representation.

The agenda for the meeting with students is included at Annex J of the QAA handbook.

The meeting is normally chaired by the review coordinator, who will introduce the subject specialist reviewers and provide a brief summary of the review method. S/he will outline the purpose of the meeting and will emphasise the importance of transparency of the review process. The dialogue with students will normally start with a question to establish on what basis the students were selected to attend the meeting.

The subject review facilitator should not attend this meeting. Throughout the meeting, students should be given opportunities to raise points not covered by the agenda.

The review team may also wish to meet recent former students who are able to give either an informed overview of the provision or of the quality of the graduates, and employers or other representatives from the profession.

As a general rule, departments/schools should aim to involve students as much as possible in the preparations for the visit, from the drafting of the self-evaluation onwards.

The aide-memoire can be used as an agenda for a preliminary meeting with the student group which is to meet with the reviewers. At least one such meeting should be organised well in advance of the visit.

Note that the students should be a representative sample from across courses and years, and that reviewers will expect to be provided with a list of the students’ names and courses.

It is also important to note that the arrangements for staff:student liaison and provision of feedback to students on assessed work will be of significant interest to the reviewers.

It is therefore essential that whatever your institution’s staff:student liaison procedures are, they are in place and well documented and that means exist of advising the student body of action taken (or not, as the case may be) as a result of discussions at staff:student liaison meetings.

Reviewers will expect to see evidence of consistent and timely feedback on assessed work and they will be keen to check the perceptions of students in this respect.

Of course this is not just about the effective operation of systems, important though that is. You need to be able to demonstrate a genuine awareness of the concerns of students in the department/school. If you do not know what the broad opinions of the student body are then you can certainly expect to be surprised by what reviewers will discover when they meet the students.

Note that arrangements for induction, the admissions process and students’ views on open days and publicity material may also be sought.

Learning resources

Reviewers also gather evidence through direct examination of the student learning resources. Reviewers normally visit the facilities made available to the department/school, and may observe students and staff using specialist IT or other equipment in the course of normal teaching and learning activities.

In looking at library provision, reviewers may undertake catalogue searches or request access to online facilities. In evaluating the quality of learning resources, reviewers’ direct observations of facilities are considered alongside evidence from student work, written documentation, meetings with relevant staff, and meetings with students. The emphasis is on access and use of facilities by law students.

The review of learning resources may inform judgements in relation to other aspects that are affected by the quality of available resources.

Reviewing the evidence

Each review includes a number of meetings between members of the institution and reviewers to consider the various aspects of provision related to quality and standards.

The review coordinator is responsible for ensuring that the review team meets sufficiently often to consider the accumulating evidence and the team’s findings. If such meetings take place at the institution, the team may find it helpful to include the facilitator, who can provide factual information relevant to the team’s discussions. However, the facilitator may not attend team meetings or parts of meetings at which direct discussion of judgements takes place.

Telephone or e-mail contacts between the team and the department/school may be used to request information or to give notice of issues that the reviewers might wish to explore.

All reviewers will be expected to identify, share, consider and evaluate evidence related to the programmes under scrutiny. Reviewers will be expected to evaluate how the accumulating evidence compares with the evidence provided by the subject provider in the self-evaluation, and to test the strength of the evidence adduced to support the judgements.

A final meeting of the reviewers will be used to review any additional evidence, to agree the particular strengths and weaknesses in relation to both standards and quality, to finalise the judgements, and to determine precisely what is to be reported.

Final observations on the subject review visit

• remember that it is vital to ensure that everyone in the department/school is available to meet reviewers at relevant times during the various visits which will take place
• this is a formal process with a distinctive, specified purpose, and the reviewers will need to maintain a professional distance, albeit coupled with a cordial atmosphere. Do not expect, for example, that they will wish to have any form of social interaction with the department/school.
• ensure that there are no indiscretions on the part of department/school staff – comments about other staff or other parts of the institution help no one and can disrupt the visit
• attitude is everything – a positive, confident, assertive, and open approach to the visit makes the task of the reviewers easier and offers the best prospect of a successful result

Last Modified: 30 June 2010